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Background / Context: 

High school graduation rates remain unacceptably low in the U.S., especially among 
disadvantaged youth (Chapman, Laird, Ifill, & KelalRamani, 2011; Stillwell, 2010), with troubling 
implications for future earnings and employment status (Bureau of Labor Statistics, 2012). Check & 
Connect (C&C) is an individualized program that pairs students with a trained mentor who closely 
monitors their progress in school and matches them with targeted academic and social supports, with 
the goal of increasing student engagement and performance in school and improving on-time 
graduation rates. C&C was originally developed to address concerns about low high school completion 
rates among students with disabilities. A strong body of prior research has demonstrated the 
effectiveness of C&C on school persistence and progression among students receiving special education 
(Sinclair, Christenson, Evelo & Hurley, 1998; Sinclair, Christenson &Thurlow, 2005; What Works 
Clearinghouse, 2006). Although C&C was developed for use with students in special education, the 
program continues to be used by some school districts with low completion rates to prevent dropout 
among general education students as well. 

The use of C&C with a broader population of students is supported by research suggesting that 
the strategies employed in this intervention can be effective at preventing high school dropout in the 
general education population. Student graduation outcomes may be improved by strengthening adult 
and student relationships (Balfanz & Legters, 2006), embedding academic support for all students into 
the regular school schedule (Dynarski & Wood, 1997), and increasing the rigor and pace of coursework 
(Kemple, 2004; Kemple et al., 2005; Kemple & Snipes, 2000). However, no rigorous studies have tested 
the efficacy of C&C for general education students at heightened risk of dropping out of high school. 

Purpose / Objective / Research Question / Focus of Study: 

The study, funded by a grant from the Institute of Education Sciences, is testing the impact of 
C&C on school engagement, attendance, and completion outcomes among general education students 
at high risk of dropping out. We used student risk indicators from Grade 8 and the first half of Grade 9 to 
select students at risk for dropping out of high school. Students within the selected high-risk sample 
were then randomly assigned within schools to participate in C&C or to a business-as-usual control 
group. The study was designed to generate rigorous evidence about the effects of C&C on engagement, 
school completion, and academic outcomes of at-risk general education students, and to document 
implementation. 

Setting: 

The study took place in the San Diego Unified School District (SDUSD), an urban district in 
Southern California. Ten comprehensive high schools in SDUSD volunteered to participate in the study 
and are described in Table 1. 

Population / Participants / Subjects: 

The study sample included 9th graders in the ten study schools with lowest probabilities of on- 
time graduation, based on their risk factors from Grades 8 and 9 (prior to study implementation). To 
identify these students, we first identified the 8th/9th grade risk factors that were most predictive of on- 
time graduation using the district's administrative data from previous cohorts of students, for whom 
high school completion outcomes were known. Next, we used data for students who were in their first 
year of high school in 2010-11 to generate a predicted probability of on-time graduation (in spring of 
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2014) for each student in the ten study schools. Students with the lowest probabilities of on-time 
graduation in their schools were recruited for study participation. Random assignment to treatment 
condition occurred after recruitment and parental consent were completed. The final study sample 
included 553 students. Table 2 presents descriptive data on the sample at baseline, including 
demographics and 8*79*^ grade risk factors. 

Intervention / Program / Practice: 

Check & Connect's theory of action centers on student engagement, and recognition that 
dropout is the culminating event of a cumulative process of increased disengagement from school that 
typically occurs over many years (Anderson, Christenson, & Lehr, 2004; Fine, 1991; Orfield, 2004). C&C 
intends to interrupt this process through early identification of observable signs of disengagement, 
followed by the provision of personalized mentoring designed to reengage students in school. Once 
reengagement begins, mentors can help students navigate the academic supports and other available 
resources that may help improve academic and behavioral outcomes. C&C has two main components: 
the check component involves monitoring indicators of student engagement (e.g., attendance, behavior, 
educational progress); the connect component has two levels of intervention: basic and intensive. All 
participating students receive basic intervention, including regular conversations about their progress in 
school and problem-solving strategies. Students who show continued signs of disengagement receive 
intensive services (e.g. facilitated tutoring, meetings between home and school, community resources) 
to encourage them to reengage in school. C&C is designed to provide support to students over multiple 
years of school. 

C&C was provided to students in the study's treatment group for three years, occurring 
continuously from the summer after Grade 9 (August 2011) through spring 2014. Five mentors hired by 
SDUSD from the local community delivered the program. Mentors received training and support from 
the model developers and were supervised by a district-based lead mentor. 

Research Design: 

The study has a block-randomized trial design, with within-school random assignment of 
students to condition. In July 2011, after identifying the study's sample of students at heightened risk of 
dropping out, students were randomly assigned within each school to either receive C&C mentoring 
(n=276) or to the business-as-usual control group (n=277). Students assigned to the treatment group 
were invited to participate in C&C for the next 3 years (through the spring of their scheduled 12th grade 
school year). Students assigned to the control group had access to support services that would be 
typically available to them, but they did not have access to C&C. This study design created two groups 
that were statistically equivalent at baseline (as shown in Table 2), permitting estimation of the impact 
of C&C with high internal validity. 

Data Collection and Analysis: 

Student Baseline and Outcome Measures 

Administrative Records. Prior to study recruitment and also at the end of each semester 
of the 2011-12, 2012-13, and 2013-14 school years, SDUSD provided the following student- 
level data for all study students who remained within the district school system: attendance; 
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course academic and citizenship grades'; credits; enrollment, transfer, or exit status with dates 
for eaeh status ehange; days enrolled and days present; and graduation status (as of June 2014 
and August 2014). Data prior to July 2011 were used as baseline information; data after this date 
are used as outeome data.^ In addition, SDUSD provided student demographie and soeio- 
economic data at baseline. At the time of this proposal, all of these data are in hand exeept final 
validated graduation data, whieh will be provided in October 2014. 

Student Survey. We measured altitudinal outcomes, including different aspects of student 
engagement, using an annual spring survey for all study students. The survey includes the 
Student Engagement Index (Appleton, Christenson, Kim, & Resehly, 2006), composed of five 
subscales that were validated by the developers: teacher-student relationships, control and 
relevance of sehool work, peer support for learning, future aspirations and goals, and family 
support for learning. 

Implementation Measures 

Implementation data ineluded monitoring logs completed by the mentors for each student 
on a monthly basis. The mentors used the logs to record their contacts with students, sehool 
personnel, parents, and others; student indicators of risk (e.g., attendance, grades, and behavior 
referrals); designated students as “high risk” or not, and recorded speeific referrals/interventions 
used, frequency, and duration. Additionally, both mentors and treatment group students were 
surveyed eaeh fall regarding the amount and type of C&C serviees delivered. The spring student 
survey also gathered data about the range of support services available to treatment and eontrol 
group students, allowing us to describe the serviee contrast. 

Analysis Methods 

This paper will report results from two sets of analyses: (1) impact analyses for all study 
outcomes; and (2) descriptive analyses of implementation data. To carry out the main impact analysis, 
we estimate multiple regression models as follows: 

X 

Y,^a + m + Y.P.X,+e, ( 1 ) 

1 

where Yi represents an outcome Y for student i, Ti is the treatment indicator (0 for control and 1 
for treatment), fio is the program effect, and Xxi is a vector of pre-random-assignment background 
characteristics whose aceompanying regression eoefficients are fix- These variables Xxi inelude 
student race and ethnicity; gender; predicted probability of on-time graduation (described in 
Population above); and a vector of school dummies (fixed effects) identifying each individual 
school from which the participating students are sampled. These dummy variables remain 
unehanged even if students moved to a different school. Lastly, si is a random error term. For 
some outcomes, K will be binary variables (coded 0/1, representing e.g. whether or not a student 
remains enrolled in school). In these eases, we estimate Equation I with a logit link function. 

Findings / Results: 

Impact Analyses 


^ Citizenship grades reflect the teacher's impression of the general behavior, attitudes, values, and habits of an individual 
student in the school community, and are provided for most classes in addition to traditional academic grades. 

^ At the end of each term, data on enrollment status are collected for all students in the initial study sample. For students 
enrolled in any SDUSD school, other administrative records data were collected directly from SDUSD. For students 
enrolled in charter schools in the San Diego area, we attempted to collect the other administrative records data from the 
charter schools. For students not enrolled in SDUSD or local charter schools, other administrative records were not 
collected from the exit date onward. 
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This paper will report impacts of C&C on a full set of student outcomes, including annual 
measures of attendance, student enrollment status, course-taking (course failures, on-track course 
taking, and credit accumulation), citizenship grades, pass/fail status on state exit exams, self-reported 
measures of student engagement, and graduation. Thus far, analyses conducted on interim outcomes 
measuring student engagement and progress toward on-time graduation in 2011-12 and 2012-13 have 
yielded no evidence of statistically significant program impacts. Results from 2012-13 are shown in Table 
3. However, while no indication of treatment effects emerged in the first two years of implementation, 
it is possible that with an increased emphasis on credit recovery, outcomes in the final year of 
implementation may be different for C&C students than their control group counterparts; this will be a 
main focus of the proposed paper. 

Implementation Analyses 

C&C was designed to be delivered primarily through bi-weekly meetings between mentors and 
students. Based on the monitoring forms submitted, the mentors met with students approximately 2.3 
times per month in Year 1 and 2.4 times per month in Year 2 on average during the academic year, 
although there was variation in the average number of meetings by mentor as illustrated in Table 4. 
However, in annual student surveys, approximately two thirds of students in the treatment group 
reported meeting with mentors at least once every two weeks. 

As noted in the intervention description above, the C&C model is designed to offer "basic" 
services to all treatment students and additional "intensive" services to students deemed to be at risk 
based upon the mentor's review of the "check" data for that month. Table 5 summarizes the percentage 
of treatment students receiving intensive services (according to the mentors), the percentage 
designated high risk (by the mentors), and the percentage who were both designated high risk and 
receiving intensive services, based on the monitoring forms from May 2013 (for the sake of illustration). 
During this month, 92 percent of treatment students were designated high risk; of those, 75 percent 
were provided intensive services. However, of the other 8 percent of treatment students who were not 
designated high risk (not shown in Table 5), nearly two-thirds also received intensive services from 
mentors during that month. 

The data suggest that mentors met with students on their caseload about as much as expected. 
Mentors also seemed to have "checked" students' data as intended by the intervention model, but they 
also added their own judgment to decisions about the level of intensity for the intervention provided. 
Most students with whom mentors met received what mentors described as "intensive" intervention. 
That is, mentors did not differentiate services as might be expected based on "check" data exclusively. 
Analyses in this paper will further explore mentoring strategies used by C&C mentors in this study, by 
mentor and by students' incoming level of risk and setting (i.e., attending original school, other school 
within the district, or charter schools within the local geographic area). 

Conclusions: 

Given the grim prospects for students who drop out of high school, or graduate high school 
underprepared for the demands of college or work, the need to identify effective intervention models is 
high. This study is expanding the evidence base to address the effects of a highly touted mentoring 
model used with general education students with increased risk of academic failure. While interim study 
results did not detect impacts on student engagement and academic outcomes, analyses presented in 


SREE Spring 2015 Conference Abstract 


4 



this paper will focus on impacts after the full three years of treatment, including graduation outcomes. 
Implementation analyses indicate that C&C was implemented as intended for some, but not all students, 
with less variation (i.e., individualization) than expected in the intensity and types of interventions 
provided to treatment students via their mentors. This paper will complement the other papers in the 
proposed symposium by providing a detailed picture of one prominent mentoring model and how it was 
implemented with the student population on which this study focused. The discussion will compare and 
contrast mentoring strategies used with different types of students with varying academic and non- 
academic needs. 
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Appendix B. Tables and Figures 


Table 1. School Characteristics of Study Schools and All SDUSD High Schools 



All SDUSD high 


Study Schools 

schools 

Characteristics 

Average Percent 

Average Percent 

Race/Ethnicity 



White 

12% 

23% 

African American 

16% 

13% 

Hispanic 

56% 

48% 

Asian 

14% 

12% 

Native American 

1% 

<1% 

Eligible for free or reduced-price lunch 

83% 

66% 

ELL 

29% 

33% 

Proficient on English Language Arts State Assessment (2010) 

37% 

53% 

Proficient on Mathematics State Assessment (2010) 

13% 

50% 


Mean (SD) 

Mean (SD) 

Average predicted probability of on-time graduation for 2010- 
2011 ninth graders (class of 2014) 

0.82 (0.05) 

0.85 (0.08) 


Table 2. Characteristics of Check & Connect Study Sample Overall and by Condition 


Student Characteristic 

Overall 

Treatment 

Control 

p-value 

Percent limited English proficient [n = 437) 

72% 

71% 

73% 

0.702 

Percent female [n = 545) 

Percent racial/ethnicity group [n =545) 

48% 

46% 

50% 

0.281 

Percent African American 

12% 

12% 

13% 

0.901 

Percent White 

8% 

9% 

7% 

0.497 

Percent Asian 

6% 

6% 

7% 

0.475 

Percent Hispanic 

Percent students who were absent 10% or of 

72% 

72% 

72% 

0.934 

enrolled days in Grade 9 [n =545) 

Percent students whose behavior grades at Grade 9 
had at least one "Need to improve or unsatisfactory" 

23% 

24% 

22% 

0.524 

[n =545) 

Percent students who failed any course in Grade 9 [n 

90% 

91% 

89% 

0.472 

=545) 

99% 

99% 

99% 

0.203 

Percent students who failed Algebra [n =545) 

79% 

80% 

79% 

0.965 

Average predicted probability of graduation [n = 

0.55 

0.56 

0.55 


545) 

(0.15) 

(0.15) 

(0.16) 

0.892 


Note. Differences in student characteristics by condition were tested using a model that accounts for the clustered 
data structure. Numbers in parentheses are standard deviations. 
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Table 3. Comparison of Treatment and Student Engagement and Progress Toward On-Time 
Graduation in 2011-12 And 2012-13 


Treatment Predicted Control Predicted 

Interim Outcomes, 2012-13 Mean (SD) Mean (SD) n p 

In 11*^ Grade in Fall 2012-13 

0.445 (.235) 

0.461 (.230) 

459 


Total Credits Earned as of July 2013 

24.7 (5.9) 

25.2 (5.8) 

383 


Credits Earned in On-Track ELA Courses 

0.318 (.152) 

0.274 (.127) 

382 


Credits Earned in On-Track Math Courses 

0.099 (.093) 

0.126 (.101) 

382 


Two or More Course Failures 

0.548 (.146) 

0.585 (.140) 

374 


Ever Passed Fligh School Exit Exam 

0.622 (.129) 

0.628 (.121) 

552 


Attended School 90% of Days or More 

0.801 (.107) 

0.838 (.091) 

422 


Two or More Unsatisfactory Citizenship 

0.365 (.133) 

0.351 (.126) 

383 


* = p < .05; ** =p< .01; *** =p< .0001; 


Table 4. Average Number of Meetings per Month for Mentors and Their Students Between December 
2012 and July 2013 and September 2013, by Mentor 



Number of Meetings per Month (out of Spring 2013 Caseload) 

Mentor 

Dec 

Jan 

Feb 

Mar 

Apr 

May 

Jun 

Jul 

Sep 

A 

1.6 

1.7 

1.8 

2.1 

1.5 

1.4 

1.5 

1.3 

1.6 

B 

1.3 

2.0 

1.8 

2.0 

1.9 

2.2 

1.8 

1.4 

1.7 

C 

2.3 

2.8 

2.4 

2.5 

2.3 

2.9 

2.1 

2.1 

2.6 

D 

2.4 

2.8 

3.4 

3.6 

3.2 

3.5 

3.5 

3.6 

3.8 

E 

1.9 

3.0 

3.0 

3.6 

1.0 

3.7 

5.0 

3.3 

0.0 

Total 

1.9 

2.5 

2.5 

2.8 

2.0 

2.8 

2.8 

2.4 

1.9 


Table 5. Percentage of Students Receiving Intensive Services, Designated High Risk, and Both 
Designated High Risk and Receiving Intensive Services in May 2013* 


Mentor 

Percent of Students 
Receiving Intensive 
Services 

Percent of Students 
Designated High 
Risk 

Of Students Designated High 
Risk, Percent Receiving 
Intensive Services 

A 

57% 

89% 

59% 

B 

80% 

93% 

69% 

C 

88% 

98% 

85% 

D 

91% 

75% 

92% 

E 

88% 

94% 

90% 

Total 

86% 

92% 

75% 


^ The denominator is the total number of students in a mentor’s caseload as of spring 2013 
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